Rejoinder: Latent variable graphical model selection via convex optimization

1. Introduction. We thank all the discussants for their careful reading of our paper, and for their insightful critiques. We would also like to thank the editors for organizing this discussion. Our paper contributes to the area of high-dimensional statistics which has received much attention over the past several years across the statistics, machine learning and signal processing communities. In this rejoinder we clarify and comment on some of the points raised in the discussions. Finally, we also remark on some interesting challenges that lie ahead in latent variable modeling. Briefly, we considered the problem of latent variable graphical model selection in the Gaussian setting. Specifically, let X be a zero-mean Gaussian random vector in R p+h with O and H representing disjoint subsets of indices in {1,. .. , p + h} with |O| = p and |H| = h. Here the subvector X O represents the observed variables and the subvector X H represents the latent variables. Given samples of only the variables X O , is it possible to consistently perform model selection? We noted that if the number of latent variables h is small relative to p and if the conditional statistics of the observed variables X O conditioned on the latent variables X H are given by a sparse graphi-cal model, then the marginal concentration matrix of the observed variables X O is given as the sum of a sparse matrix and a low-rank matrix. As a first step we investigated the identifiability of latent variable Gaussian graphical models—effectively, this question boils down to one of uniquely decomposing the sum of a sparse matrix and a low-rank matrix into the individual components. By studying the geometric properties of the algebraic varieties of sparse and low-rank matrices, we provided natural sufficient conditions for identifiability and gave statistical interpretations of these conditions. Second , we proposed the following regularized maximum-likelihood estimator to